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Abstract. There is a contradiction in 

the literature about the effect of REACT 
strategy on science achievement. This 
study aims to resolve this contradiction by 
determining the effect of REACT strategy 
on student science achievement and the 
factors affecting this strategy by integrat- 
ing findings obtained from both qualita- 
tive and quantitative studies. The study 
was conducted using a mixed-research 
synthesis by including 19 quantitative 
and 10 qualitative studies. Data obtained 
from quantitative findings were combined 
using a meta-analysis method, and data 
from qualitative findings were combined 
using a thematic synthesis method. It 

was attempted to explain the variance 
between studies included in the meta- 
analysis by using analytic themes derived 
from the thematic synthesis. As a result of 
the meta-analysis, teaching based on the 
REACT strategy was found to have a strong 
effect on science achievement (ES = 1.041, 
95% Cl: 0.7876 to 1.2948). The thematic 
synthesis yielded four descriptive themes, 
“teaching-learning process in the REACT 
strategy”, “learning outcomes in the REACT 
strategy’, “limitations of the REACT strat- 
egy’, and “recommendations for practice’. 
Teaching based on the REACT strategy was 
found to contribute largely to the learn- 
ing process and learning outcomes. Some 
limitations were found in practice, and 
recommendations were determined to 
address these limitations. 
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Introduction 


One of the most important objectives of modern education is to enable 
students to transfer what they learn at school to daily life and to gain knowl- 
edge and skills they could use in solving problems they encounter. For this 
purpose, Many approaches are used in the education process. One of these 
approaches is context-based learning. Contextual teaching is defined as an 
approach in which the contexts and practices adopted in science teaching 
are used as the starting point in developing scientific ideas (Bennett et al., 
2003). This approach focuses on connections students make between sci- 
ence and situations they encounter in daily life (Demircioglu et al., 2019). 
Thus, this approach enables students to establish a relationship between 
what they have learned in the past and present (Gunter, 2018) and to put 
together the facts taught independently from each other through these 
situations (Gilbert, 2006). 

One of the strategies used for applying a contextual teaching in science 
education is the REACT strategy. This strategy, which was first introduced 
by Crawford (2001), consists of five stages, relating, experiencing, applying, 
cooperating, and transferring. In his work, “Teaching Contextually’, Crawford 
(2001) explains these stages as in the following: (1) In the relating stage stu- 
dents learn using their prior knowledge and experiences. For this purpose, 
teachers expose students to problems they will encounter in daily life. (2) 
In the experiencing stage students learn by doing and discovering. For this 
purpose, teachers may use problem-solving techniques, laboratory activi- 
ties, or models relating to the problem state presented. (3) In the applying 
Stage students learn by applying the concepts they have discovered. For 
this purpose, teachers motivate students to understand concepts by hav- 
ing them complete realistic and life-related exercises. (4) In the cooperating 
stage, students learn by sharing the knowledge they gain and by establishing 
communication with each other. For this purpose, in the application stage, 
teachers conduct exercises or activities together in groups formed under 
student leadership. (5) In the transferring stage students use the knowledge 
they have learned by adapting it to different situations (new and original). 
For this purpose, teachers enable students to transfer the knowledge they 
have learned by means of presenting different situations that they have never 
encountered in the lesson before. 
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Many studies examine the effectiveness of the REACT strategy in the literature. Most studies on teaching based 
on REACT strategy are quantitative and employ a quasi-experimental design (e.g., Demircioglu et al., 2019; Majid 
& Rohaeti, 2018; Putri et al., 2019; Rifni, 2014). One of the most important weaknesses of the quasi-experimental 
design is obviously the non-random assignment of subjects to the experimental and control groups (Creswell, 
2012, p. 322). Although using existing groups in this design is advantageous, it threatens the internal validity of 
the study. Nonetheless, the quasi-experimental design is used the most in educational research. Considering the 
literature, Most quasi-experimental studies examining the effectiveness of REACT strategy are conducted in science 
discipline area (science, physics, chemistry, and biology courses; e.g., Dori et al., 2018; Karsli Baydere & Aydin, 2019; 
Keskin & Cam, 2018; Ultay, 2012). However, some studies also use this design in mathematics (Quadri et al., 2019; 
Widada et al., 2019) and language education (Rifni, 2014; Rohayati, 2013). In addition to the quantitative studies in 
the literature, there are also qualitative (e.g., Asmahasanah et al., 2018; Sari & Darhim, 2020) and mixed-methods 
studies (e.g., Karsli Baydere & Aydin, 2019; Karsli & Yigit, 2017; Yildirim & Gultekin, 2017) that explore the views of 
study participants about the REACT strategy applications. 

Studies examining the effect of the REACT strategy have found that teaching based on this strategy improves 
students’ conceptual understanding (Demircioglu et al., 2019; Jelatu et al., 2018; Karsli & Yigit, 2017; Ultay, 2012), 
increases science achievement (Ingram, 2003; Keskin & Cam, 2019; Majid & Rohaeti, 2018; Putri et al., 2019), posi- 
tively affects their attitudes towards the course (Asmahasanah et al., 2018; Ingram, 2003), heightens motivation 
(Ingram, 2003; Ozbay & Kayaoglu, 2015; Yildirim & Gultekin, 2017), and improves problem-solving (Widada et al., 
2019), creative thinking (Qadri et al., 2019), and writing skills (Rifni, 2014). Contrarily, some studies report that RE- 
ACT strategy does not have a significant effect on science achievement or conceptual understanding (e.g., Gdkalp 
& Adem, 2020; Ultay et al., 2014). Thereby, one could say that there is a disagreement in the literature about the 
effect of REACT strategy on science achievement. 

No meta-analysis study was found in the literature to resolve the contradictions regarding the effect of REACT 
strategy on science achievement. However, two systematic reviews were found on context-based learning (é.g., 
Bennett et al., 2003; Ultay & Ultay, 2014). Ultay and Ultay (2014) conducted a content analysis of context-based 
studies in physics according to certain themes. They evaluated 32 studies included in their content analysis study 
according to rationale, purpose, method, findings, and recommendations. As a result of this systematic review, 
they found that a vast majority of studies were conducted to fulfill meaningful learning by creating contexts that 
would attract students’ attention. A systematic review study was conducted by Bennett et al. (2003) to determine 
the contribution of contextualizing science and associating it with science, technology, and society to students’ 
understanding of science and developing positive attitudes towards science. The study found that the context- 
based approach endorses students’ motivation towards, helps them develop a positive attitude, and does not affect 
their scientific knowledge negatively. The common feature of both studies conducted by Ultay and Ultay (2014) 
and Bennett et al. (2003) is providing general information about the existing studies on context-based learning in 
the literature. By contrast, this study aims to resolve the present conflict concerning the effect of REACT strategy in 
the literature and to determine what characteristics a REACT Strategy-based intervention should have. This study 
is considered significant in resolving the existing contradiction in the literature and contributing to future studies 
from a conceptual and methodological perspective. Therefore, answers were sought to the following questions: 

1. What is the effect of teaching based on the REACT strategy on science achievement? 
2. What are the views and experiences of students about teaching based on the REACT strategy? 
3. What factors influence the effectiveness of teaching based on the REACT strategy? 


Research Methodology 


A mixed research synthesis method was used in this study. Mixed research synthesis is a systematic literature 
review method that aims to combine findings obtained from qualitative and quantitative studies on the same 
subject area (Sandelowski et al., 2006). This method consists of three stages (Harden, 2010): (1) combining quan- 
titative study findings using a meta-analysis method, (2) combining qualitative study findings using a thematic 
synthesis, and (3) comparing the quantitative and qualitative syntheses results. The Preferred Reporting Items for 
Systematic Review and Meta-Analysis Protocols (PRISMA) Statement was consulted in the meta-analysis part of 
this research. The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews, 
meta-analyses and reporting systematic reviews of other types of research, particularly evaluations of interven- 
tions (Moher et al., 2009). 
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Searching the Literature 


Studies included in this study were retrieved from the Google Scholar, EBSCO, ERIC, SSCI, YOK dissertation center, 
and PROQUEST databases. Searching began in February 2020, including a complementary search that took place 
in May 2021. The literature search was performed by two different researchers using similar keywords. Researchers 
came together at certain intervals and developed different search strategies by comparing the studies they found. 

Keywords including “REACT Strategy’, “Context-Based Instruction’, “Context-Based Learning’, “Contextual Strategy’, 
and “Teaching Contextually” were used when performing the search. Further, the reference sections of the retrieved 
studies were examined to access relevant studies. As a result of searching the literature, 344 records that used REACT 
strategy were retrieved. 


Inclusion/Exclusion Criteria 


As this study was a mixed research synthesis, separate inclusion criteria were determined for meta-analysis 
and thematic synthesis studies. Studies to include in the meta-analysis should (1) have been conducted between 
2001-2020 in science and mathematics, (2) examine the effect of REACT strategy on students’ performance, (3) use 
experimental design, (4) report quantitative data needed for calculating effect sizes, and (5) use parametric or non- 
parametric statistical methods. Literature reviews, conference papers and weak experimental design-based research 
were not included in the meta-analysis. 

A total of 344 records were retrieved through the literature search. Of these records, 81 were identified as duplicate 
copies. The titles and abstracts of 263 records remaining after excluding duplicate copies were examined considering 
the inclusion criteria of the meta-analysis. Although 203 of these records were related to context-based teaching, they 
were not designed according to REACT strategy. Therefore, 60 studies were screened for suitability. Of these studies, 41 
weak experimental design and non-experimental studies, 4 studies that did not report necessary data to analyze the 
effect size and 9 studies out of science and mathematic fields were excluded from the meta-analysis. Consequently, 
19 experimental studies applying the REACT strategy were identified. Nineteen studies revealing the effectiveness 
of REACT strategy-based teaching practices on students’ science achievement were included in the meta-analysis. 

The total number of students in the studies included was 1298. Of these students, 653 formed experimental 
groups, and 645 formed control groups. The characteristics of 100% (f= 19) studies included in the meta-analysis 
were quasi-experimental. In quasi-experimental studies, the experimental groups received REACT strategy-based 
instruction, whereas the control groups received instruction based on what the curriculum required. However, 21.1% 
(f= 4) of studies supported REACT strategy using a different teaching approach, method, and technique (2 supported 
by computer, 1 supported by explanation, and 1 supported by conceptual change text). To measure the science 
achievement of experimental and control groups, 52.6% (f= 10) of studies used science achievement test, 31.3% (f 
= 6) used subject concept test, and 15.8% (f= 3) used conceptual understanding test. Moreover, 52.63% (f= 10) of 
studies were conducted in primary, 21.05% (f= 4) in secondary, and 26.32% (f= 5) in higher education. Also, 89.48% 
(f= 17) of studies were conducted in science (physics, chemistry, and biology) and 10.52% (f= 2) in mathematics. Of 
73.68% (f= 14) of studies published as articles and 26.32% (f= 5) as dissertations, 78.95% (f= 715) used parametric tests 
(t-test; ANOVA) and 21.05% (f= 4) used non-parametric tests (Mann-Whitney U test). 

As such, studies included in the thematic synthesis should (1) have been conducted between 2001-2020 in science 
and mathematics, (2) explore the views of individuals participating in REACT strategy practices, (3) report participants’ 
direct expressions or themes and code revealed in the study, and (iv) use a qualitative or mixed-methods research de- 
sign. Studies exploring views without the participation of individuals in REACT strategy practices were not included in 
the thematic synthesis. Accordingly, 10 studies including qualitative data (qualitative or mixed-methods studies) were 
included in thematic synthesis. When the characteristics of 10 studies included in thematic synthesis were examined, 
5 were mixed-methods (quantitative + qualitative) and the other five qualitative studies (action research, case study, 
phenomenology, etc.). A total of 155 participants’ views on REACT strategy practices were obtained in these studies. 


Evaluating the Quality Studies 
Quality assessment of experimental studies included in the meta-analysis was performed using an assessment 


system for experimental studies in health sciences developed by Pluye et al. (2009). In this assessment system, the 
quality of quantitative experimental studies is recommended to be evaluated according to three criteria. These cri- 
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teria are as follows: (1) reporting the application procedure and randomization in sampling, (2) concealing the group 
information (random assignment of the groups), and (3) ensuring the validity/reliability of the data and the absence 
of data loss. It is recommended to provide 1 to a study that matches these criteria and 0 if it does not match. However, 
0.5 points were also given to studies because the criteria could be partially met in social sciences. The quality score 
was calculated using the (total score obtained / 3) *100 formula. Pluye et al. (2009) do not determine a percentage 
value regarding the quality/low quality of studies, whereas, in the context of this study, studies with a quality score 
of 50% were considered high quality, and studies below this percent were considered low quality. Nineteen studies 
included in the meta-analysis were determined to be of sufficient quality. 

Furthermore, a checklist of 12 criteria proposed by Harden et al. (2006) was used to determine the quality of 
studies included in the thematic synthesis. Although there is no general rule, Harden et al. (2006) consider studies that 
meet less than 7 out of 12 criteria as “low’, studies that meet 7-9 as “moderate”, and studies that meet 10-12 as “high” 
quality. They recommended including studies of moderate and high quality in the thematic synthesis. As such, it was 
determined that 10 studies included in the thematic synthesis had “moderate” and “high” quality scores. 


Data Extraction 


The two coding forms were prepared for coding the quantitative studies. In the first form, studies were coded ac- 
cording to author names, dependent variables (science achievement, attitude, and self-efficacy), research design (quasi 
or weak experimental designs), characteristics of the study groups (education level), intervention features (course type; 
application duration), measurement tools (achievement test; performance test), and data analysis tests (parametric; non- 
parametric). A second form was prepared to extract the quantitative data from studies that met the inclusion criteria. 
Author names, post-test mean scores (for the studies reporting a non-significant difference regarding pre-test mean 
scores between the experimental and control groups), standard deviation, sample size, and dependent samples t-test 
scores were extracted using this second form. In case of the missing mean score, standard deviation, and sample size, the 
result of independent samples t-test and sample size or F test results and sample size were collected. In non-parametric 
studies, the Mann-Whitney U and Wilcoxon W values and the experimental and control group sample sizes were coded. 

As such, the qualitative data were extracted from studies that met the inclusion criteria. Direct quotes referring to 
students’ experiences and views or code tables regarding the factors facilitating or disincentivizing the REACT strategy- 
based teaching were extracted. Researchers coded the qualitative and quantitative data, and the inter-coder reliability 
was computed as 100%. 


Synthesis of Quantitative Findings 


A meta-analysis method was used to determine the effectiveness of the experimental procedure. Since the 
sample sizes in studies included in the meta-analysis were greater than 25, Cohen's d was used as an effect size index 
(Card, 2012, p.93). The mean effect size was calculated according to the random-effects model, as the studies were 
collected from the literature (Borenstein et al., 2009, p.86). The calculated effect size was interpreted as “weak” if it was 
0-0.20,“small” if 0.21-0.50,“moderate’ if 0.51-1.0, and “strong” if greater than 1.0 (Cohen et al., 2007, p.521). Studentized 
residuals and Cook's distance were examined to determine the presence of potential outliers among the effect sizes 
calculated (Viechtbauer & Cheung, 2010). Studies with a studentized residual greater than the 100 x (1—0.05 / (2 x 
k)) the percentile of a standard normal distribution were considered as potential outliers (k denotes the number of 
studies). Studies with a Cook's distance larger than the median plus six times the interquartile range of the Cook’s 
distances were accepted to be influential. 

Moreover, a heterogeneity test was conducted to determine the presence and size of the variance between the 
effect sizes. To determine the amount of heterogeneity, the DerSimonian-Liard estimator (DerSimonian & Liard, 1986) 
was used. The size of heterogeneity was interpreted according to the Pindex. An Pindex of 25% is considered as low, 
50% moderate, and 75% high heterogeneity (Higgins et al., 2003). Further, the prediction interval was calculated to 
determine the distribution of true effect sizes at a 95% confidence level (Riley et al., 2011). If the heterogeneity test 
was significant, moderator analysis was performed to determine the source of heterogeneity. In order to determine 
the presence of publication bias in studies, the funnel plot was examined by conducting Egger’s Regression Inter- 
cept Test, Begg and Mazumdar Correlation Test, and Rosenthal’s Fail-Safe N test. The analysis was carried out using R 
(version 4.0.0) (R Core Team, 2018), the metaphor package (version 2.4.0) (Viechtbauer, 2010), and the Meta package 
(version 4.18.0). 
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Synthesis of Qualitative Findings 


A thematic synthesis method was used in analyzing the qualitative findings. This analysis method has three stages 
(Thomas & Harden, 2008): In the first stage, the direct quotations or basic concepts extracted from the qualitative 
studies are read and coded line by line. In the second stage, codes obtained from the first stage are compared and 
grouped according to their similarities and differences. Hence, the descriptive themes are created. In the third stage, 
however, hypotheses relating to what characteristics agood REACT strategy-based teaching practice should possess 
are created by comparing the descriptive themes. This way, the analytical themes are produced. 


Cross-Study Synthesis 

At this stage, moderators were created according to the analytic themes obtained from the thematic synthesis. 
Studies with quantitative findings were classified according to these moderators and categorical moderator analysis 
was conducted to determine whether the effect sizes yield significant differences, that is, to determine whether the 
moderators derived from the thematic synthesis are significant. Here, it was attempted to determine factors affecting 
the effectiveness of REACT strategy-based teaching practices on science achievement. 
Research Results 

Synthesis of Quantitative Findings 


A forest plot of the effect sizes from 19 studies included in the meta-analysis study is given in Figure 1. 


Figure 1 
Forest Plot of Meta-analysis Results 


Study Hedges g [95%-Cl] zval P-value 

1.Demirciogluetal,2019 2193[1.591-2.795] 7.143 <0.01 i -—f- 
2 Gil et al, 2017 1.164 [ 0.607; 1.721] 4.099 <0.01 —- 

3. Majid & Roh, 2018 0.932[ 0.417: 1.447] 3544 <0.01 i 

4 Ultay, 2012 0.877 [ 0.489; 1.265] 4429 <0.01 is 

5 Ultay et.al, 2015 -0.037 [-0.539: 0.465] -0.145 0.89 

6 Ingrid, 2003 1.060 [ 0.621; 1.499] 4.732 <0.01 a 

7 Ultay & Alev, 2017 1.766 [ 1.113; 2.419] 5303 <0.01 i—- 
8.Aktas, 2013 0.642[ 0.136: 1.148] 2488 0.01 —- 

9 Karsh & Saka, 2017 1.134 [ 0.466; 1.802] 3.326 <0.01 —- 
10.Kirman Bilgin, 2015 0.615[ 0.215; 1.015] 3.015 <0.01 ss 
11.Seving, 2015 0.861 [ 0.391: 1.331] 3.587 <0.01 ff 

12 Keskin & Cam, 2019 0.815[ 0.280; 1.350] 2.985 <0.01 —- 
13.Ginter, 2018 1.427 [ 0.980: 1.874] 6.259 <0.01 ; 2 
14.Kirman Bilgin et al, 2017 1.166 [0.747; 1.585] 5449 <0.01 7 2 
15.Ayvacl et.al., 2016 1.198 [ 0.467; 1.929] 3212 <0.01 —i-— 
16.Ercoban, 2018 0.692 [ 0.082: 1.302] 2.225 0.03 —i- 

17 Puspita, 2016 2.389 [ 1.703; 3.075] 6.826 <0.01 | —- 
18.Gdkalp & Adem, 2020 0.007 -0.483: 0.497] 0.028 0.98 . on 

19. Karas, 2020 1.341 [ 0.780; 1.902] 4.689 <0.01 “H- 
Random effects model 1.041 [ 0.787; 1.295] > 


WSs 


Predictioninterval [-0.035; 2.118] 

Heterogeneity: /” = 78%, t° = 0.2436, x5, = 82.29 (p < 0.01) 

Test for overall effect: z = 8.04 (p = 0.01) > «# -7 UO FT 2&2 @ 
Favors Control Favors Experiment 





According to Figure 1, the study with the largest effect size (ES = 2.389) is a thesis by Puspita (2016), and the 
one with the smallest effect size (ES = -0.037) was an article by Ultay et al. (2015). Studentized residuals and Cook’s 
distance were examined to determine whether these studies were outliers. According, none of the studies in the 
context of this study had studentized residuals greater than the threshold value of + 3.0078, which required con- 
sidering a study as an outlier. 

The estimated common effect size based on the random-effects model was 1.041 95% Cl [0.787, 1.295]. As seen in Figure 
1, the common effect size differed significantly from zero (z = 8.04, p < .01). This common effect size is “strong” according to 
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Cohen et al’s (2007) classification. This d value is associated with a U3 value of 84.1% (Cohen, 1988, p.22). It means that the 
average student receiving REACT-based instruction (experimental group) scored higher on achievement tests than 84.1% 
of students not receiving such instruction (control group). 

According to Figure 1, the result of the heterogeneity test is significant (Q .,, =82.29, p < .05). The I’ index is 78%, which 
means that there is a high amount of heterogeneity among the studies. A 95% credibility/prediction interval for the true 
effect sizes is given by -0.035 to 2.118. Hence, although the common effect size is estimated to be positive, in some studies 
the true effect size may in fact be negative. 

Since the heterogeneity test was significant (p < .05), moderator analysis was conducted to determine the source of 
heterogeneity. For this purpose, studies included in the meta-analysis were classified according to the school level (secondary 
school, high school, university), course type in which the study was carried out (science, mathematics), the measurement 
tool used for measuring science achievement (achievement test, concept and conceptual understanding tests), types of 
statistical methods used in analyzing the collected data in studies (parametric, non-parametric), and contamination status of 
studies (contaminated/uncontaminated) considering whether a study was contaminated using another teaching approach, 
method, or technique together with the REACT strategy. According to the moderator analysis conducted based on these 
categorical variables, none of the selected moderators were significant (p < .05). Accordingly, the variance existing between 
studies was not due to the level of schools where studies took place, the type of the course, the type of analysis tests used 
in studies, and supporting studies with other approaches and methods in addition to the REACT strategy. 

Moreover, a meta-regression was performed to determine whether the application duration (week) significantly 
predict the effect sizes. However, two studies, not reporting the application duration, were not included in the analysis. 
As a result of meta-regression, the model tried to explain the variance between the effect sizes with the application time 
was non-significant (p < .05). Accordingly, no significant relationship may exist between the application duration of the 
experimental procedure and the effect sizes. 


Publication Bias 


According to Card (2012, p. 262), one of the best methods for determining the publication bias is to include unpub- 
lished studies in meta-analysis and test whether studies show significant differences according to their publication status 
(thesis versus article). If there was a non-significant difference between effect sizes considering their publication status (p 
>.05), then there may not be a publication bias. This meta-analytic study included 5 thesis (unpublished) and 14 articles 
(published) studies. A categorical moderator analysis was conducted to determine if there was a significant difference ac- 
cording to the effect sizes of studies. As a result of the analysis, the common effect of the articles was calculated as 1.059 
95% Cl [0.733, 1.385] and that of the theses as 1.004 95% Cl [0.573, 1.436]. No significant difference was found between 
these two common effects (p > .05). According to this finding, one could argue that there may not be a publication bias. A 
funnel plot of the estimates is shown in Figure 2. Both the rank correlation and the regression test did not indicate potential 
funnel plot asymmetry (p = .064 and p = .087, respectively). 


Figure 2 
Funnel Plot of the Effect Sizes 
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Rosenthal’s Fail-Safe N test was performed to determine whether the estimated mean effect was robust. 
According to the result of this test, 1375 studies with effect sizes of zero should be included in the meta-analysis 
to make the estimated mean effect non-significant. If the number of studies required is more than 10 plus five 
times the number of studies included in the meta-analysis (threshold value = 5k + 10; kis the number of studies), 
one could conclude that the mean effect is robust and is not a product of publication bias (Rosenthal, 1979). 
Accordingly, considering studies included in the meta-analysis, the threshold value is (5 x 19 + 10) 105. Given 
the number of studies required (1375) is much larger than the threshold value, one could argue that the mean 
effect size calculated is strong and not the product of publication bias. 


Synthesis of Qualitative Findings 


Ten studies of “moderate” and “high” qualities were included in the meta-analysis. The qualitative findings of 
these studies, in other words, the direct quotations in the findings or the code definition tables of the researchers 
were extracted and imported into one of the Computer Assisted Quantitative Data Analysis Software (CAQDAS). 
In the first stage of the thematic synthesis, the qualitative data entered into the program were read and coded 
line by line. In the second stage, codes with similar code definitions were combined under the same codes by 
expanding the code definition. Then, the new codes were compared considering their similarities and differ- 
ences and codes with similar features were gathered under four themes including (1) Teaching-learning process 
in the REACT strategy, (2) Learning outcomes in the REACT strategy, (3) Limitations in the REACT strategy, and 
(4) Recommendations for practice. Teaching-learning process in the REACT strategy: Research & Inquiry, Mean- 
ingful Learning, Organizing the Lesson, Relating to Daily Life, Learning by Having Fun, Cooperative Learning, 
Enabling Active Participation; Learning outcomes in the REACT strategy: Higher-Order Thinking Skills, Motiva- 
tion, Retention, Scientific Process Skills, Positive Attitude, Academic Self-Confidence; Limitations in the REACT 
strategy: Domain-Specific Knowledge and Skills, Group Work Skills, Crowded Classroom, Grade Level, Daily Life 
Problems, Equipment, Time, Exam Anxiety, Material Complexity; Recommendations for Practice: Considering 
the Readiness, Using Up-to-date Examples, Considering the Grade Level, Active Participation in Activities, Taking 
responsibility for one’s own learning. 

Some limitations, influencing the teaching-learning process and the learning outcomes of students, were 
determined. These limitations were gathered under the theme of the limitations of the REACT strategy. Accord- 
ingly, the following codes were more prominent: (1) students’ lack of domain-specific knowledge and skills, (2) 
students’ inadequate skills to work in groups, (3) crowdedness of the intervention class, (4) unsuitability of the 
grade level, (5) lack of equipment for the activities, (6) unsuitability of the presented everyday life problems 
to students’ level, (7) anxiety caused by the exam given at the end of the process, (8) complicatedness of the 
materials presented, and (9) allocation of insufficient time for the activities. 

Some recommendations were obtained for activities to be conducted in the REACT strategy. These recom- 
mendations may help overcome the limitations of the REACT strategy and may also affect the learning outcomes 
of students. The following codes emerged under the theme of the recommendations for practice: (1) taking the 
responsibility for one’s own learning, (2) ensuring active participation in activities, (3) considering the level of 
readiness, (4) considering the grade level, and (5) teachers’ use of up-to-date examples. 

In the third stage of the thematic synthesis, the two researchers discussed peculiarities that a good REACT 
strategy intervention should have considered the descriptive themes and the following analytic themes (hy- 
potheses/recommendations) emerged as a result of this discussion: 

1. Determine the duties and responsibilities of group members in group work clearly under the guid- 

ance of the teacher. 

2. Conduct introductory activities to develop students’ skills in working collaboratively 

3. Conduct a pilot implementation regarding the REACT strategy 

4. Teacher provides the materials to use 

5. Uncover and develop students’ domain-specific basic knowledge and skills before starting the im- 
plementation. 
Determine real-life problems to use in the REACT strategy that suit the student level. 
Conduct formative assessment at the end of the REACT strategy rather than summative assessment. 
Prepare materials to use in accordance with the principles of material development. 
The REACT strategy is not suitable for every grade level (suitable to use in grade seven and above). 
0. The number of students in the intervention class is not high (24 persons or less are convenient). 
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Cross-Study Synthesis* 


In order to determine whether the analytical themes derived from the thematic synthesis were covered 
in the experimental studies, the two researchers examined the experimental studies according to the analytic 
themes. The duties and responsibilities of students were clearly mentioned when applying the REACT strategy 
only in one study '*. None of the studies included in the meta-analysis conducted introductory activities in order 
to develop students’ group work skills. Likewise, no activity was conducted to uncover and develop students’ 
domain-specific knowledge and skills. Four studies included in the meta-analysis conducted pilot studies of 
the REACT strategy before applying this strategy to the experimental group®'®'"'’. Of 19 studies included in 
the meta-analysis, in 16 of them the equipment required for the REACT strategy was provided by teachers '7* 
6,7,8,9,10,11,12,13,14,15,16, 18,19" 

In all studies included in the meta-analysis, students’ science achievement was determined by an exam at 
the end of the process. Therefore, none of the studies determined students’ science achievement in the REACT 
process by process-oriented measurement tools (such as portfolios). Of these studies, only one reported that 
the materials used during the implementation process of the REACT strategy were prepared in accordance with 
the material development principles®. Relatedness of problems used in the REACT strategy to daily life and their 
suitability to the level of students is important in terms of implementation. Considering studies included in the 
meta-analysis, the selected problems in 17 out of 19 studies were related to daily life and suited student level '?*° 
678310,11,12,13,1419,16,18,19 Dascriptive themes revealed that the REACT strategy may not be suitable for all grade levels, 
especially under grade seven. In 10 studies included in the meta-analysis, it was found that the REACT strategy 
was applied to sample groups at grade seven and above '*?*°°71''3:18, Similarly, the size of classes where the 
REACT strategy is implemented affects the effectiveness of this strategy, as well. Although no recommendation 
relating to ideal class size emerged from the thematic synthesis, the idea that ideal class size should be 24 or 
less was determined as a proper recommendation. However, the class size policy report of the England National 
Education Union that recommends 24 pupils in case of mixed age class (https://neu.org.uk/advice/class-sizes), 
and the finding showing that “the average class in OECD countries at the lower secondary level has 23 students” 
(OECD, Education at a glance report, 2016) were considered when determining the ideal class size. Besides, the 
guideline on class size published by Canada Ontario National Education also required the boards to ensure 
that the maximum average class size in the primary division did not exceed 24 (The Class Size Guideline of the 
Ontario Ministry of Education, 2009). In this context, a convenient class was considered to be 24 for this study. 

It was found that in three of the studies included in the meta-analysis, the experimental groups had a class 
size of 24 and below?®">'®, Lastly, the implementation of the REACT strategy with computer-assisted instruction was 
found time-consuming. Two experimental studies applying computer-supported REACT strategy were identified®"®. 

Categorical moderator analyses of recommendations emerging from the analytical themes were conducted 
to determine whether they explain the variance between studies included in the meta-analysis and to determine 
whether these recommendations are functional. According to the results of the categorical moderator analyses 
relating to recommendations applied in studies 

of 11 analytical themes derived from the thematic synthesis, only one (the 11 analytical theme) significantly 
accounted for the variance between studies (p < 0.05). Accordingly, the mean effect size of the computer-aided 
REACT strategy was 0.321 [-0.301, 0.943], while the mean effect size of the studies that applied this strategy 
without computer support was 1.126 [0.874, 1.378]. According to this finding, applying computer-supported 
REACT strategy is time-consuming and therefore may cause a decrease in students’ science achievement. 

The 9" and 10" analytical themes could be considered as continuous moderators rather than categorical 
ones. In other words, the 9" analytical theme could be rephrased as: “the effect size increases as the grade level 
of the class in which the REACT strategy is applied increases”. Given the REACT strategy becomes more difficult 
to implement as the grade level decreases, we could expect a decrease in the effect size calculated for science 
achievement. Similarly, the 10 analytical theme could be rephrased as: “the effect size decreases as the class size 
increases in the intervention class”. As the class size increases, the applicability of the REACT strategy decreases, 
which may lead to a decrease in the effect size calculated for science achievement. Therefore, meta-regression 
analyses were conducted to determine the practicability of these themes. As a result of the meta-regression 
analysis, the models tested to explain the effect sizes related to science achievement using grade level and size 
of the intervention class were non-significant (p > 0.05). Therefore, one could argue that the size of the grade 
level of the class which received the intervention, and the class size does not have a significant association with 
the effect sizes related to science achievement. 
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Discussion 


The main purpose of this study is to resolve the conflicting findings in existing studies in the literature on the 
effect of the REACT strategy on science achievement. In addition to studies reporting the positive impact of the 
REACT strategy on science achievement in the literature (e.g., Ingram, 2003; Keskin & Cam, 2019; Majid & Rohaeti, 
2018; Putri et al., 2019), some studies report that it has no significant effect (e.g., Gdkalp & Adem, 2020; Ultay et al., 
2015). This meta-analysis determined that teaching according to the REACT strategy can have a strong effect on 
the science achievement of students. However, the prediction interval indicated that teaching based on the REACT 
strategy may not be effective for all populations. Therefore, one could say that this study may have partially resolved 
the existing conflict in the literature. The reasons for the effectiveness of the REACT strategy were determined 
using a thematic synthesis method. Accordingly, it was determined that in classes where the REACT strategy was 
applied, students took active participation in the lesson, had a collaborative and meaningful learning experience, 
related what they learned to everyday life, and learned while having fun. At the end of this learning process, it 
was revealed that students’ motivation increased, and they had developed a positive attitude towards the course. 
Supporting this result, a systematic literature review by Bennett et al. (2003) determined that the context-based 
approach endorses students’ motivation towards science and helps them develop positive attitudes. Therefore, 
one could say that the REACT strategy is a teaching strategy that contributes to increasing students’ interest and 
motivation towards the course and developing their attitudes. 

Another finding in this study revealed that the experimental studies included in the meta-analysis are highly 
heterogeneous in terms of their effect sizes. Studies included in the meta-analysis were coded according to their 
characteristics to explain the variance between them (i.e., course type, school level, test type, and contamination 
status) but these codes could not explain the source of the variance. Therefore, analytical themes were obtained by 
combining the qualitative studies using thematic synthesis that express what characteristics a sound experimental 
intervention should have (Thomas & Harden, 2008). Of 11 analytical themes, only one contributed to explaining 
the variance between studies. This analytic theme suggested that “Computer-supported REACT strategy is time- 
consuming.’When the experimental studies were examined considering the implementation status of this theme, 
two studies (Aktas, 2013; Gdkalp & Adem, 2020) carried out using a computer-assisted REACT strategy were identified. 
The moderator analysis revealed that the effect sizes of studies conducted by applying the REACT strategy using 
computer support were smaller compared to those of other studies conducted without providing such support. 
In other words, it was determined that computer-supported delivery of the REACT strategy might negatively af- 
fect the science achievement of students. One of the reasons behind this may be the fact that computer-assisted 
activities take time and require effort. Undoubtedly, computer-assisted activities contribute to students’ learning 
processes and facilitate their learning, but in such activities, necessary information should be given before stu- 
dents can use computer-assisted activities. In computer-assisted implementations of comprehensive and intense 
activities such as REACT and STEM, if students do not have a previous learning experience, the learning process 
may sometimes be negatively affected. In their study, Brain et al. (1999) reported that computer-assisted learning 
process did not provide students an enjoyable environment as in conventional learning and this process took a 
lot of time according to the students’ opinions. 

Three out of the 11 analytical themes were not observed in studies included in the meta-analysis. The first one 
of these themes was “Conducting activities for developing students’ skills in working collaboratively.’ In the fourth 
stage of the REACT strategy, teachers conduct the exercises or activities under student leadership in the implemen- 
tation stage (Crawford, 2001). Therefore, when students’ collaborative working skills are inadequate, they may not 
fulfill their tasks in group work as desired. This, in turn, affects the effectiveness of the REACT strategy. The second 
theme was “Uncovering and developing students’ domain-specific basic knowledge and skills before starting the 
implementation.’ This analytical theme requires students’ cognitive readiness for the activities to be performed. 
Cognitive readiness can be defined as the mental preparation (knowledge, skill, ability, etc.) that individuals need 
in establishing and maintaining a competent performance (Morrison & Fletcher, 2002). In the educational environ- 
ment, however, this condition requires students to know basic concepts about the activities to be performed and 
be able to use the tools and equipment required for activities. For example, a student's ability to use a microscope 
or to prepare an experimental setup is related to cognitive readiness. Since students have to find solutions to 
real-life problems in the REACT strategy (Crawford, 2001), students’ state of having a sufficient level of cognitive 
readiness is important for the effectiveness of this strategy. The third analytical theme was “Conducting formative 
assessment at the end of the REACT strategy rather than summative assessment.” None of the studies included in 
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the meta-analysis employed alternative measurement tools to measure students’ achievement (performance task, 
portfolio, etc.). Of summative assessment techniques, achievement tests and conceptual tests were used instead. 
Supporting these findings, through a systematic review, Ultay and Ultay (2014) found that studies on contextual 
learning did not prefer alternative assessment techniques such as portfolios and worksheets to collect data. Studies 
have determined that using summative assessment techniques increases exam anxiety in students (Kumas, 2015; 
Ultay, 2014). High examination anxiety also negatively affects student achievement (Kumandas & Kutlu, 2014). 


Conclusion and Implications 


The REACT strategy is teaching that contributes to increasing students’ science achievement. However, 
one should pay attention to the fact that students have adequate domain-specific knowledge and skills, have 
collaborative working skills, and that students’ achievements are measured using formative assessment tech- 
niques instead of summative assessment techniques to ensure the effectiveness of this strategy. However, when 
the experimental intervention is carried out with computer support, its efficacy may become limited, as the 
implementation may take time. 

However, both the categorical moderator analysis and the meta-regression revealed that the grade level 
and class size were not associated with achievement. According to this result, the REACT strategy may be also 
suitable for grade levels lower than the seventh grade (grades 6 and 5) and be successfully applied in classrooms 
with class sizes of more than 24. Similarly, the implementation duration of the experimental procedure (class 
hour/week) was also a non-significant predictor of the effect size relating to science achievement. When the 
primary experimental studies were examined, the implementation duration of the REACT strategy was observed 
to range between 1-9 weeks. However, the lengthiness or the shortness of the implementation duration have 
no relationship with the effect of the REACT strategy on achievement. Therefore, the implementation duration 
could be considered an unimportant variable in experimental studies. 

In this study, attempting to explain the variance between the effect sizes calculated by meta-analysis with 
the help of the analytical themes derived from the thematic synthesis is the strong aspect of this study, but it 
also has some limitations. One of the first of these limitations is including studies using non-parametric tests 
together with studies using parametric tests which may negatively affect the generalizability of the overall ef- 
fect size. The second limitation is related to the 11" analytical theme that accounted for the variance between 
studies included in the meta-analysis. According to this analytical theme, the effect size yielded by studies that 
applied a computer-assisted REACT strategy in experimental interventions differed significantly from those 
studies that did not use such support. However, the computer-supported implementation of only two out of 
19 experimental studies limits the generalizability of this result. There is a need for more experimental studies 
in this regard. The third limitation may be related to studies contaminated using different teaching approaches, 
methods, and techniques. Although there was no significant difference in the moderator analysis according to 
the contamination status of the studies (four out of 19 studies were contaminated with different teaching ap- 
proaches, methods, and techniques), it may affect the overall effect size obtained. 


Endnote 


*Numbers on studies given in the Cross-Study Synthesis section are the sequence number of studies 
included in the meta-analysis. 
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